Summary

Phase C (user-authorized after the Phase B null result): candidates whose text_rank saturates the min(text_rank, 1) clamp collapse to byte-identical imputed primaries, so unrelated documents matching the same terms tie exactly and release order falls to arbitrary chunk-id — the live mechanism behind the four fast-path headroom cases (canary #53/#54: identical rr@10 0.20/0.25/0.33/0.33).

The fix, per the dedicated design investigation (full consumer map + envelope proof in the design record):

• saturationTailUnit(tr) = (tr−1)/(tr−1+0.5) — pure, strictly monotone, set-independent (one candidate's value never depends on the rest of the pool).
• Table facts (S2): similarity carries a ≤0.02 tail into its dead (0.92, 0.94) cap band; hybridScore stays byte-identical — so selection scores, every gate/threshold (all ≤0.82), and the second-stage trigger are provably unchanged. similarity is exactly the tie-break key after hybrid/releaseRankScore in the release comparators, so only exact-tie resolution changes.
• Lexical chunks (S1): signature-guarded hybrid lift into the dead (0.48, 0.5) band (similarity === 0 && lexical_score && text_rank > 1); the sub-0.5 truthful-score contract holds (comment updated at the buildRetrievalDiagnostics gate; RC9 fabrication inventory amended).
• Sub-knee is byte-identical: test fixtures now derive from the helper — text_rank 0.45 still yields exactly the historical 0.755/0.795 pair, keeping every pre-Phase-C guard's premise intact.

Rejected designs (recorded for review): per-query min-max normalization (set-dependent; full-envelope spread gives raw ts_rank enough authority to reproduce the #118 recall-regression mechanism), rank-position tiers (ill-defined across interleaved variant streams; set-dependent), full-range log rescale (moves sub-knee values across the 0.62–0.82 gate ladder — direct recall risk).

Scope honesty: the tail resolves ties where the second-stage rerank does not engage — matching the live evidence (patient-safety, opioid, flowchart pools; the engaged ciwa-threshold pool sorts by position-derived releaseRankScore first). S3 (document-lookup respread) and S4 (memory-card confidence ties) are deferred as C-PR-2 candidates, evidence-gated on the post-merge canary.

Verification

• Targeted vitest 121/121: rag-fast-path-ordering (11/11 — 6 new Phase C tests), retrieval-selection, rag-routing, rag-answer-fallback, ranking-tuning, rag-second-stage-ranking
• Red proof executed: with the tail removed (old formulas restored), the discriminating test fails exactly as designed — id-order seats the distractor first — plus the envelope test; restored code passes 11/11
• Tie-conservation guard: genuinely equal text_ranks still produce identical primaries and fall to the coverage comparator (fix(rag): break saturated selection ties by query-term coverage, not rank score #987 contract preserved)
• npm run test — 3025 passed / 1 failed: the known container-only pdf-extraction-budget artifact (baselined on clean main)
• npm run lint + npm run typecheck + prettier — clean
• npm run eval:retrieval:quality (36/36 required) — cannot run locally (live keys are Actions-only). Live validation = one post-merge eval-canary dispatch (~$1–2, user-authorized cap) against the banked Enhance clinical safety and governance #54 baseline: doc/content recall must stay 1.0, zero per-case rr regressions; success signal = rr lift on the non-engaged headroom cases. I will dispatch it immediately on merge and report the pair.

Risk and rollout

• Risk: retrieval ordering (clinical-path). Three realistic regressions and their catches: (R1) ts_rank outranks the truly-relevant doc at a former tie — authority capped at 0.02 (below every boost; coverage comparator sorts earlier in selection), caught by the tie-conservation test + the canary pair's zero-regression/recall-1.0 gates; (R2) gate/threshold drift — impossible by construction (movement confined to dead bands above 0.92/0.96/0.48; sub-knee identity; movers only rise) and pinned by envelope unit tests + the routing/fallback suites; (R3) S1 topScoresClose trigger adjacency (a 0.03–0.04 hybrid gap crossing 0.04) — low frequency, caught by the second-stage suites + the canary; fallback is halving the S1 tail or deferring S1 to C-PR-2, both one-liners.
• Rollback: single-commit revert (pure helpers + two expression sites + one map call; no schema/config/cache/workflow surface — RAG_RANKING_CONFIG untouched, cache entries age out at TTL).
• Provider or production effects: none from the diff; the validating canary dispatch is explicitly user-authorized (≈$2–4 spent of the $20 cap).

Clinical Governance Preflight

• Source-backed claims still require linked source verification before clinical use — answer path/verification untouched
• No patient-identifiable document workflow introduced or expanded
• Supabase target remains Clinical KB Database (sjrfecxgysukkwxsowpy) — no schema/SQL change (the SQL clamp stays; the spread is app-side)
• Service-role keys and private document access remain server-only
• Demo/synthetic content remains clearly separated from real clinical sources
• Source metadata, review status, and outdated/unknown-source behavior remain conservative — governance tie-break direction re-verified by the existing outdated-twin test; synthetic-similarity origin tags unchanged (confidence stays capped at medium)
• Deployment classification/TGA SaMD impact: retrieval ordering refinement at exact ties only; conservative failure behavior unchanged

Notes

• Post-merge sequence: canary dispatch vs Enhance clinical safety and governance #54 baseline → if green, regenerate the ranking snapshot from the post artifact and re-run the tuner before any Phase B weight staging resumes (its recommendations were fitted to pre-spread features).
• Ledger rows appended: the Phase B no-adopt pair record and this change (content commit 7572c7f).

🤖 Generated with Claude Code

https://claude.ai/code/session_01EXsJcLrbZUXwnBeG91cVo9

Generated by Claude Code

Summary by CodeRabbit

• New Features
  • Improved retrieval ranking for saturated text and table-fact matches, providing more precise result ordering.
  • Enhanced tie handling to prioritize raw relevance when candidates have similar scores.
• Bug Fixes
  • Preserved existing retrieval behavior and score contracts while improving discrimination among closely ranked results.
• Documentation
  • Updated retrieval guidance and review records with the latest scoring behavior, validation results, and known synthetic-text pathways.